Skip to content

Match repo file lookups case-insensitively - #12

Merged
hsnice16 merged 1 commit into
mainfrom
fix/case-insensitive-path-lookup
Aug 23, 2026
Merged

Match repo file lookups case-insensitively#12
hsnice16 merged 1 commit into
mainfrom
fix/case-insensitive-path-lookup

Conversation

@hsnice16

Copy link
Copy Markdown
Owner

Summary

  • Repo file lookups now match case-insensitively, so readme.md, Readme.md, and README.MD all count
    as a README. Eight tracked repos — including vercel/next.js, expressjs/express, and nestjs/nest
    were recorded as having no README at all.
  • next.js was under-scored by 18.5 points; JetBrains/kotlin and directus/directus likewise.
  • /methodology now states the matching rule so maintainers can see why a file counts.

Motivation

Scores are the product. firstExisting used existsSync(join(repo, candidate)), which is exact-match.
README / LICENSE / CONTRIBUTING casing varies genuinely in the wild, so on a case-sensitive filesystem
those files read as missing.

The scheduled rescore runs on ubuntu-latest, so the wrong values are the ones in data/rank.db and on
the live leaderboard. macOS masked it locally — existsSync("README.md") happily matches Readme.md
there — which is why it survived this long.

Not a roadmap item; an unplanned correctness bug found while scoping on-the-fly scoring.

Changes

  • lib/scoring/signals/helpers.ts — case-folding resolver behind firstExisting, plus resolveRelative
    and resolveAllRelative. Exact spelling wins so an exact match is never shadowed by a differently-cased
    sibling; entries are sorted so a genuine README.md + readme.md collision resolves identically every run.
  • Ten signals moved off raw existsSync(join(repo, …)).
  • resolveAllRelative dedupes by resolved path — a candidate list carrying two spellings of one file must
    not count twice. Without this, Makefile + makefile in dev_env's list would have counted as two
    artifacts and pushed the signal from 0.7 to 1.0.
  • Dropped now-redundant candidates (makefile, Tests); folded gemini-md's hand-rolled case-insensitive
    scan into the shared resolver.
  • Case-insensitive glob regexes for .csproj / .cabal / .nimble / .mdc / .ya?ml.
  • app/methodology/page.tsx — matching rule stated in the Signals panel and in the FAQ JSON-LD.
  • AGENTS.md — Conventions bullet requiring the resolvers over raw existsSync, new test file listed,
    "Adding a signal" points at it.

Testing

  • bun run test — 179/179 (was 166; +13 covering case-insensitivity, resolved-path dedupe, and repo-root escape).
  • bun x tsc --noEmit, biome check . — clean (3 pre-existing noImgElement warnings in badge components, untouched).
  • bun run score . end-to-end; next build compiles, /methodology still static.
  • macOS cannot exercise this bug, so verification ran on a case-sensitive APFS volume against a fixture
    spelled the way next.js spells its files. Old logic: existsSync false for all of README.md / LICENSE /
    LICENSE.md / CONTRIBUTING.md / CONTRIBUTING. New logic: readme, license, contributing all pass=1.
  • The new tests assert the on-disk spelling comes back rather than the candidate spelling — that is what
    makes them fail pre-fix on a case-insensitive filesystem too.
  • Scored 25 real clones: 66.7 ms/repo, no regression.

README / LICENSE / CONTRIBUTING casing varies in the wild (readme.md,
Readme.md, README.MD). Exact-match lookups scored those files as missing
on case-sensitive filesystems, so Linux CI and a macOS dev box disagreed
on the same commit — and the deployed numbers were the wrong ones.
vercel/next.js, expressjs/express, nestjs/nest and five other tracked
repos were recorded as having no README at all; next.js was under-scored
by 18.5 points.

Signal path lookups now go through case-folding resolvers in helpers.ts.
resolveAllRelative dedupes by resolved path so a candidate list carrying
two spellings of one file (Makefile / makefile) cannot count twice — on
a case-insensitive filesystem that had been inflating dev_env to 1.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Aug 23, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agent-friendly-code Ready Ready Preview Aug 23, 2026 5:56am

@github-actions

Copy link
Copy Markdown
Contributor

Agent Friendly Code — score diff

Overall: 91.0 → 91.0 (±0.0)

Per model

Model Base Head Δ
Claude Code 95.3 95.3 ±0.0
Cursor 85.5 85.5 ±0.0
Devin 94.8 94.8 ±0.0
GPT-5 Codex 96.3 96.3 ±0.0
Kimi CLI 96.1 96.1 ±0.0
Gemini CLI 85.5 85.5 ±0.0
Aider 83.1 83.1 ±0.0
OpenHands 85.4 85.4 ±0.0
Pi 96.6 96.6 ±0.0

No signals changed between base and head.


Posted by agent-friendly-action.

@hsnice16
hsnice16 merged commit 9371d6c into main Aug 23, 2026
4 checks passed
@hsnice16
hsnice16 deleted the fix/case-insensitive-path-lookup branch August 23, 2026 05:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant